Steam installation:
Look for tModLoader in the Steam Store and install.

Manual installation:
Make a 'tModLoader' folder next to the Terraria install folder and place the .zip contents in it. Launch the provided 'tModLoader' file instead of the regular one.
Run Terraria through the provided files. If it doesn't run, you might need to give execute permissions to 2 files. If so, open a terminal in the install directory and run the following 2 commands:
    chmod +x tModLoaderServer
    chmod +x tModLoader

Why are there EXE files?:
Don't worry about it, you didn't download the wrong thing.

Run tModLoader:
Just launch the provided tModLoader file (tModLoader) or launch through Steam. If you see Mods or Mod Browser on the main menu in Terraria, you have successfully installed tModLoader.

Uninstall/Switch back to un-modded Terraria:
In Steam, right click tModLoader in your library -> manage -> uninstall
If you installed manually, simply remove the tModLoader folder.
Note: If you want to play un-modded Terraria or join an un-modded Terraria server, you can't use tModLoader.

Dedicated tModLoader server:
If you are installing tModLoader onto a machine that will be running a dedicated server, you'll need to install the un-modded Terraria server first: http://terraria.org/server/terraria-server-1353.zip  After installing, you can follow the manual install instructions to copy the tModLoader files into the folder you installed the server into. More info on running a dedicated server can be found at https://github.com/tModLoader/tModLoader/wiki/Command-Line#dedicated-host .